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PREFACE 


This  report  covers  research  conducted  by  the  Flying  Training  Division  of  the  Air  Force 
Human  Resources  Laboratory  between  November  1976  and  August  1979.  Full  support  of  this 
project  was  provided  by  the  82d  Flying  Training  Wing,  Williams  AFR,  Arizona  85224.  The  Air 
Force  Flight  Test  Center,  Edwards  AFR,  California,  also  provided  full  support  for  the 
collection  and  reduction  of  the  cine-theodolite  aircraft  data.  The  research  project  was 
conducted  in  support  of  Project  1123,  Flying  Training  Development,  Mr.  James  F.  Smith, 
Project  Scientist;  Task  112303,  Exploitation  of  Flight  Simulation  in  Undergraduate  Pilot 
Training  (UPT),  Mr.  Robert  R.  Woodruff,  Task  Scientist;  Work  Unit  11230332,  Terrain 
Visual  Cue  Requirements  for  Flight  Simulation,  Capt  George  H.  Buckland,  Principal 
Investigator  and  Eric  G.  Monroe,  Associate  Investigator. 

The  authors  would  like  to  extend  special  thanks  to  Majs  Samuel  Hannan,  Michael 
Moorman,  Jay  Paulsen,  Capts  James  Brucia,  Peter  Drechsler,  Lee  Lesher,  Charles  Reddick, 
Ronald  Wetters,  Lts  Michael  Cranford,  Kimball  Jenson,  Roy  Poole,  and  Reginald  Willich  for 
their  dedication  as  subjects  for  the  study. 
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FLIGHT  SIMULATOR  RUNWAY  VISUAL  TEXTURAL 
CUES  FOR  LANDING 


L  INTRODUCTION 


The  lack  of  adequate  flare  and  final  touchdown  visual  information  cues  has  been  a  longstanding 
criticism  of  flight  simulators  in  general  (Armstrong,  1968,  1970;  Bray,  1973;  Chase,  1971;  Crane, 
1962;  Dyda  &  Lew,  1963;  Sorum  &  Fister,  1974;  Valverde,  1968;  Wempe  &  Palmer,  1970).  A 
typical  fault  often  mentioned  is  the  lack  of  adequate  textural  information  (Armstrong,  1968)  in  the 
visual  scene.  Improvements  in  this  area  might  provide  better  cues  for  depth  perception.  It  is  not 
clear  what  the  best  cues  for  depth  perception  are;  however,  textural  information  may  be  such  a  cue. 
The  flexibility  of  computer  image  generation  (CIG),  which  permits  rapid  variation  of  visual  scene 
content,  appeared  ideal  for  testing  the  utility  of  textural  patterns  as  landing  cues. 

One  major  difficulty  in  assessing  the  adequacy  of  a  flight  simulator  visual  scene  is  the  problem  of 
specifying  the  different  aspects  of  the  visual  environment.  Figure  1  shows  the  W  illiams  AFB  runway 
from  a  T-37  cockpit  at  three  different  points  on  the  glideslope.  Note  the  runway  tire  tracks  which 
appear  in  the  last  photo.  Although  this  visual  textural  cue  is  not  readily  apparent  from  a  distance,  it 
appears  to  become  a  powerful  cue  for  judging  depth  and  motion  as  the  pilot  approaches  it.  Many  of 
the  textural  cues  in  the  runway  touchdown  zone  are  of  an  irregular  nature,  such  as  the  tire  tracks  on 
the  runway,  and  they  are  therefore  difficult  to  vary  or  specify  in  an  orderly  parametric  fashion. 
However,  unless  pilot  performance  or  training  effectiveness  can  somehow  be  shown  to  vary  along 
some  dimension  of  textural  detail,  it  is  difficult  to  convincingly  demonstrate  that  simulated  runway 
texture  has  actually  contributed  to  improved  pilot  performance.  Also,  some  generalizable  definition 
of  textural  requirements  for  flight  simulation  in  general  is  needed  for  decisions  on  cost-effective 
designs  of  future  flight  simulators.  Binary  judgements,  such  as  “our  simulated  tire  tracks  did  or  did 
not  help,"  are  not  as  useful  in  this  regard.  Ultimately,  simulated  tire  tracks  may  be  used,  but  only 
after  they  have  been  related  to  a  more  general  dimension  of  visual  texture.  Also,  there  may  be  less 
costly  ways  to  simulate  runway  textures  than  the  edge-consuming  simulated  tire  tracks.  A  grid 
pattern  superimposed  on  the  touchdown  zone  area  appeared  to  be  a  simple  way  to  vary  runway 
texture  along  a  dimension  of  coarseness,  given  the  present  visual  image  generation  capabilities  of  the 
Advanced  Simulator  for  Pilot  Training  (ASPT)  at  the  Air  Force  Human  Resources  Laboratory/ 
Operation  Training  Division  (AFHRLA3T). 

Assessing  a  flight  simulation  visual  scene  in  terms  of  its  adequacy  for  touchdown  and  landing 
can  be  pursued  in  a  number  of  ways,  ranging  from  student  training  effectiveness  to  experienced  pilot 
performance.  The  criticism  of  touchdown  visual  information  has  often  been  stated  by  experienced 
pilots  and  demonstrated  by  their  simulated  aircraft  vertical  velocity  being  excessive  at  touchdown 
(Armstrong,  1968;  Chase,  1971).  Thus,  it  was  decided  to  initially  investigate  this  phenomenon  (i.e., 
excessive  velocity)  in  the  ASPT  by  using  experienced  pilots  who  could  be  assumed  to  have  reached  a 
stable  level  of  performance  in  the  T-37.  This  approach  permitted  assessment  of  the  relative 
differences  in  pilot  performance  in  response  to  the  different  runway  texture  patterns;  the 
performance  of  the  experienced  pilots  served  as  a  stable  basis  for  comparing  the  relative  efficacy  of 
the  different  runways.  Additional  data  collected  on  six  of  the  pilots  during  actual  T-37  landings  were 
used  to  compare  pilot  landing  performance  in  the  simulator  to  pilot  performance  during  actual 
aircraft  landings. 
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IL  METHOD 


Experimental  Design 

The  basic  simulator  experimental  design  was  a  two-factor,  repeated  measures  design  with  all 
pilots  being  tested  under  all  seven  levels  of  the  first  factor  and  both  levels  of  the  second  factor.  The 
first  factor  consisted  of  six  different  daytime  runway  types  plus  one  night  runway.  The  second  factor 
was  whether  there  was  a  1000-foot  runway  overrun  marked  with  chevrons  prior  to  the  runway 
threshold  for  the  daytime  scene  and  whether  there  were  touchdown  zone  landing  lights  (TD-Zone 
lights)  for  the  night  scene.  The  two  independent  variables  of  runway  type  and  runway  overrun/TD- 
Zone  lights  were  presented  in  a  limited  random  order,  with  each  pilot  landing  on  each  runway  type 
five  times  with  the  overrun/lights  and  five  times  without  the  overrunAights.  The  experimental 
sessions  were  divided  into  five  missions  with  each  mission  consisting  of  14  landings  in  which  each 
runway  type  occurred  once  with  the  threshold/TD-Zone  lights  and  once  without.  The  runway  types 
were  presented  in  a  random  order  for  each  mission  of  14  landings. 

Aircraft  Data 

The  T-37  aircraft  flare  and  touchdown  data  were  gathered  at  the  Air  Force  Flight  Test  Center 
(AFFTC)  at  Edwards  AFB.  The  AFFTC  facility  includes  cine-theodolite  tracking  devices  for 
determining  aircraft  time,  space,  and  position  data  during  touchdown  or  takeoff.  Using  these  data,  it 
was  possible  to  gather  information  on  the  aircraft  path  during  the  flare  and  landing,  as  well  as  such 
touchdown  data  as  the  aircraft  vertical  velocity  and  groundspeed  at  touchdown.  During  this  exercise, 
each  pilot  performed  at  least  10  touch-and-go  landings  using  a  closed  pattern  at  Edwards  AFB. 


Subjects 

Twelve  T-37  instructor  pilots  (IPs)  from  Williams  AFB  were  used  as  subjects  for  the  flight 
simulator  portion  of  the  study.  Six  of  these  IPs  also  served  as  subjects  for  the  touchdown  data 
collected  in  the  actual  T-37  aircraft.  Each  pilot  was  required  to  be  a  qualified  T-37  IP  and  to  be 
current  in  the  T-37  at  the  time  of  the  study. 


Apparatus 

The  ASPT  was  used  for  the  simulation  portion  of  the  study  (Bell.  1974).  Standard  Air  Training 
Command  (ATC)  T-37  basic  jet  trainers  were  used  for  the  aircraft  portion  of  the  study.  The 
simulated  T-37  was  always  initialized  with  1000  pounds  of  fuel  for  each  landing.  The  actual  T-37 
sorties  began  with  full  fuel  (approximately  1800  pounds)  and  continued  touch-and-go  landings  until 
nearly  empty  (approximately  300  pounds). 

The  ASPT  consists  of  two  fully  instrumented  cockpits.  One  cockpit  is  configured  as  a  T-37 
aircraft,  while  the  other  cockpit  is  currently  used  for  A-10  flight  simulation.  The  ASPT  visual  system 
uses  seven  36-inch  monochromatic  cathode-ray  tubes  (CRTs)  to  provide  a  wraparound  visual  scene 
with  a  visual  field  of  view  of  +  110  to  -40  degrees  vertical  and  ±150  degrees  horizontal.  The  visual 
scenes  are  produced  using  computer-generated  imagery  (CC1).  Almost  any  simulated  visual  scene 
can  be  produced  with  current  ASPT  hardware  and  software  as  long  as  it  does  not  require  more  than 
2000  edges  to  display  the  scene  content.  All  of  the  objects  and  surfaces  in  the  visual  environment 
must  be  constructed  of  these  straight  line  segments  or  edges.  The  visual  imagery  is  updated  at  30 
times  per  second  in  response  to  the  aircraft  movement  through  the  simulated  environment. 
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The  ASPT  cockpits  are  mounted  on  six-degrees-of-freedom,  hydraulically  actuated  motion 
platforms.  Additional  motion  cues  are  also  provided  by  32-bellow  pneumatic  g-seats  with  variable 
tension  lap  belts.  Neither  of  the  motion  systems,  i.e.,  motion  platform  and  g-seat,  was  active  for  this 
study. 

The  ASPT  has  the  ability  to  record,  store,  and  score  various  pilot  performance  parameters 
automatically.  The  measures,  discussed  later  under  dependent  variables,  are  sampled  and  stored 
from  3.75  to  15  times  per  second,  depending  on  their  nature.  The  ASPT  can  be  preprogrammed  to 
vary  any  of  the  simulator  configurations  for  a  particular  experimental  study.  Thus,  it  was  possible  to 
present  the  different  runway  types  in  quasi-random  order  for  each  pilot.  Before  each  landing,  the 
pilot  was  initialized  on  the  glideslope  approximately  one  mile  from  the  runway  threshold  for  a 
particular  runway  type.  After  touchdown,  the  aircraft  was  reinitialized  on  the  glideslope  for  a 
different  runway  type. 


Independent  Variables 

The  major  factor  which  was  varied  consisted  of  seven  different  runway  types;  six  were  daytime 
runways  differing  in  the  amount  of  textural  detail  in  the  touchdown  zone  area,  and  the  seventh  was  a 
night-time  visual  scene.  The  night  runway  scene  was  used  to  investigate  reports  that  night  landings 
are  typically  Firmer  in  terms  of  vertical  velocity  at  touchdown.  The  environment  for  the  daytime 
runway  consisted  of  a  monochromatic  background  representing  the  ground  with  a  lighter  shade  for 
the  sky  and  the  horizon  boundary.  The  daytime  scenes  contained  no  other  scene  detail  except  for  the 
runways,  and  the  night  scene  had  scattered  point  lights  only,  in  order  to  delineate  the  ground  and 
horizon. 

Figures  2  to  7  show  the  experimental  simulated  runways  from  different  points  on  a  glide  slope. 
The  glide  slope  used  for  these  pictures  is  somewhat  steeper  than  normal  in  order  to  enhance  the 
visibility  of  the  texture  patterns.  The  pictures  were  taken  from  a  black-and-white  CRT  monitor 
which  had  a  higher  contrast  ratio  than  the  monochromatic  green  CRTs  used  for  the  simulator 
cockpit.  Figure  2  shows  the  night  runway  scene  with  and  without  the  TD-Zone  lights  at  three 
different  points  on  the  glideslope.  Figures  3  and  4  show  the  six  daytime  runways  from  the  same  point 
just  prior  to  the  runway  overrun.  Figure  3  has  the  overrun,  while  Figure  4  does  not  have  the  overrun. 
Notice  that  the  texture  patterns  are  just  barely  distinguishable  at  this  distance.  The  pilot  would  be 
making  the  decision  to  flare  over  the  beginning  of  the  overrun.  Figures  5  and  6  show  the  daytime 
runways  from  a  point  over  the  overrun  just  prior  to  the  runway  threshold.  At  this  point,  the  texture 
patterns  are  clearly  distinguishable  from  each  other,  and  the  pilot  should  be  executing  the  flare  now. 
Figure  7  shows  the  daytime  runways  from  a  point  just  over  the  runway  threshold,  thus  providing  a 
very  clear  picture  of  the  texture  patterns.  It  is  important  to  note  that  these  texture  patterns  are  never 
viewed  from  a  static  position  during  landing,  and  they  produce  a  somewhat  different  visual  sensation 
during  motion. 

The  seven  runways  started  with  a  very  basic  Bare  Bones  runway  consisting  of  a  rectangular 
runway  surface  which  was  6000  feel  long  and  150  feet  wide.  The  runway  was  outlined  by  a  stripe 
which  was  the  same  width  and  intensity  as  the  dashed  centerline  stripe  which  was  also  used  on  all 
runways.  There  was  also  a  small  dark  rectangle  marking  1000  feet  from  the  threshold,  on  either  side 
of  all  daytime  runways.  The  night  runway  had  dual  “VASI  like”  lights  marking  the  1000-foot  point. 
All  of  the  runways  had  the  same  footprint  and  position  as  the  Bare  Bones  runway.  Textural  detail 
was  merely  added  to  the  Bare  Bones  runway  to  generate  the  more  detailed  runways.  The  second 
daytime  runway,  the  “Willie”  runw  ty.  was  modeled  after  the  Williams  AFB  runway.  This  runway 
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had  the  standard  Air  Force  runway  markings  and  the  number  (30)  added  to  the  Bare  Bones  runway. 
The  third  through  sixth  runways  consisted  of  the  "Willie”  runway  with  texture  patterns  underneath 
the  runway  markings.  The  runway  markings  were  thus  not  obscured  by  the  texture  patterns.  The 
patterns  covered  the  width  of  the  runway  and  extended  1000  feet  down  the  runway  from  the 
threshold.  The  texture  patterns  varied  in  size  with  25-foot  spacing  for  runway  three,  16-foot  spacing 
for  runway  four,  8-foot  spacing  for  runway  five,  and  4-foot  spacing  for  runway  six.  Although  the 
pattern  appears  to  be  a  simple  checkerboard  pattern,  it  was  specially  designed  to  conserve  more  CIC 
edges  than  would  a  plain  checkerboard.  Figure  8  schematically  presents  the  actual  texture  pattern 
which  was  used.  The  night  runway  scene  was  modeled  after  the  Vital  111  (McDonnell  Douglas 
Corporation)  visual  scene  technology.  All  vertical  light  cues  were  removed;  however,  the  point 
lights  were  retained  to  indicate  the  location  of  the  ground  plane  and  the  horizon.  The  runway 
markings,  which  were  visible  near  the  ground,  were  the  same  as  for  the  basic  ASPT  runway. 

The  second  factor  which  was  varied  was  the  presence  or  absence  of  the  runway  overrun  for  the 
daytime  scene  and  the  TD-Zone  lights  for  the  night  scene.  The  overrun  was  1000  feet  long.  This 
factor  was  explored  because  the  decision  to  flare,  and  much  of  the  flare  itself,  occurs  over  the 
overrun.  The  chevron  markings  on  the  overrun  also  provide  textural  cues.  Although  the  approach 
lights  on  the  night  scene  are  more  comparable  to  the  runway  overrun,  the  TD-Zone  lights  were 
varied  in  the  night  scene  in  order  to  provide  an  effect  which  might  be  more  comparable  to  the 
texture  patterns  on  the  daytime  runway.  Also,  it  was  thought  that  the  absence  of  approach  lights  on 
the  night  scene  would  produce  too  great  an  adverse  effect  on  the  landing  performance  on  that 
runway. 

Dependent  Measures 

The  ASPT  simulator  has  the  capacity  to  record  and  store  almost  any  variable  which  is  generated 
during  a  flight  simulation.  The  variables  measured  are  accessed  via  performance  measurement 
software  which  accesses  the  desired  variables,  computes  the  required  parameters,  and  stores  the  data 
in  an  experimental  data  file.  Table  1  shows  the  variables  which  were  analyzed  for  this  landing  study. 
The  derived  scores  (scores  1  to  4)  are  expressed  in  percentages.  They  were  used  to  condense  some  of 
the  more  detailed  measures  in  an  effort  to  provide  more  concise  measures  for  data  analysis  and  to 
provide  performance  feedback  to  the  pilots. 

Four  derived  scores  were  used:  the  final  approach  score,  the  flare  score,  the  touchdown  score, 
and  the  total  or  overall  score.  Figure  9  shows  how  these  scores  were  displayed  on  the  in-cockpit  CRT 
after  each  touchdown.  Additional  discrete  parameters  were  also  provided  on  this  scoring  page.  The 
overall  score  is  merely  an  arithmetic  mean  of  the  other  three  scores.  The  other  scores  are  computed 
on  the  basis  of  time  within  tolerance  bounds.  These  tolerance  bounds  are  shown  in  Table  2.  The 
tolerance  bands  are  referenced  to  ideal  aircraft  flightpaths  and  parameters.  These  ideal  parameters 
are  presented  as  the  base  values  in  Table  2.  The  final  approach  score  is  thus  the  percentage  of  time 
that  airspeed,  centerline  alignment,  and  glidepath  are  all  kept  within  tolerance  limits  when  the  final 
approach  segment  was  scored.  The  final  approach  segment  was  scored  from  10  seconds  after  the 
simulator  came  off  freeze  (started  landing  sequence)  until  the  flare  scoring  started.  The  tolerance 
limits  were  set  during  preliminary  studies  using  expert  pilot  performance  as  the  criterion  for  the 
tolerance  bands.  The  primary  purpose  of  the  final  approach  score  for  this  study  was  to  keep  the  pilot 
on  the  glide  path  to  prevent  coming  in  low  due  to  the  lack  of  visual  cues  prior  to  the  runway.  For 
example,  extremely  low  approaches  have  often  been  reported  in  landing  over  water  at  night.  The 
flare  score  was  based  on  airspeed,  altitude,  pitch,  and  centerline  alignment  from  the  time  when  the 
aircraft  was  1000  feet  from  the  threshold  until  touchdown.  Since  airspeed,  altitude,  and  pilch  all 
change  during  the  flare,  equations  based  on  time  from  the  start  of  scoring  were  used  for  computing 
the  tolerance  limits  for  the  flare.  This  approach  to  flare  scoring  was  experimental  in  nature.  The 
touchdown  score  was  based  on  airspeed,  heading,  and  vertical  velocity  at  the  instant  of  touchdown. 
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RUNWAY  TEXTURE  PATTERN 


The  other  dependent  measures  consisted  of  aircraft  state  parameters  which  either  represented 
the  actual  aircraft  position  and/or  travel  through  space  versus  the  ideal  path  used  for  the  derived 
scores  (RMS  deviation)  or  else  represented  the  pilot  inputs  to  control  the  aircraft.  The  pilot  inputs 
were  monitored  in  order  to  look  at  the  smoothness  of  pilot  control.  These  scores  are  referred  to  as 
smoothness  profile  scores  (number  18  to  22)  and  are  computed  over  time  at  a  15-Hz  rate.  They 
measure  either  the  amount  (degrees)  and  force  (pounds)  of  control  movement,  i.e.,  power,  or  the 
amount  of  aircraft  change  over  time,  i.e..  Roll  and  Pitch  RMS  Rate.  The  smoothness  profile  scoring 
began  with  the  final  approach  scoring  and  continued  until  touchdown. 

All  scoring  was  stopped  when  the  aircraft  touched  down  with  greater  than  50  pounds  weight  on 
wheels.  The  touchdown  values  of  airspeed,  heading,  distance  from  threshold,  and  distance  from 
centerline  were  also  taken  at  the  point  of  weight  on  wheels  greater  than  50  pounds.  However,  the 
vertical  velocity  at  touchdown  was  taken  from  the  sample  (at  15  Hz)  just  before  weight  on  wheels 
greater  than  50  pounds  was  detected,  since  the  vertical  velocity  rapidly  approaches  zero  after  the 
wheels  touch  the  pavement.  The  vertical  velocity  figures  from  the  actual  aircraft  landings  were  also 
taken  from  the  sample  just  before  the  aircraft  wheels  touched  the  pavement.  This  was  readily 
determined  from  the  cine-theodolite  photographic  data. 


Table  l.  Performance  Measurement  Data  Items 


1.  Final  Approach  Score,  % 

2.  Flare  Score,  % 

3.  Touchdown  Score,  % 

4.  Total  (Overall)  Score,  % 

5.  Final  Aproach  Airspeed,  Knots,  RMS  Deviation 

6.  Final  Approach  Centerline,  Feet.  RMS  Deviation 

7.  Final  Approach  Glidepath,  Degrees,  RMS  Deviation 

8.  Flare  Airspeed,  Knots,  RMS  Deviation 

9.  Flare  Altitude,  Feet,  RMS  Deviation 

10.  Flare  Pitch,  Degrees,  RMS  Deviation 

1 1.  Flare  Centerline,  Feet.  RMS  Deviation 

12.  Flare  Altitude.  Feet,  MSL  at  start  of  Flare  scoring 

13.  Touchdown  Position  from  Centerline,  Feel 

14.  Touchdown  Position  from  Threshold,  Feet 

15.  Touchdown  Airspeed,  Knots 

16.  Touchdown  Heading,  Degrees 

17.  Touchdown  Vertical  Velocity,  Fl/Min 

Smoothness  Profile,  processed  at  15  Hz: 

18.  Aileron  Power,  Lb-Degrees^econd 

19.  Roll  RMS  Rate,  Degrees/5econd 

20.  Elevator  Power,  l,b-Pegrees<5econd 

21.  Pitch  RMS  Rate,  Degrees/Second 

22.  Rudder  Power.  Lb-DegTees^>econd 
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Overall  Score  43.68 
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Figure  9.  In-Cockpit  CRT  display  for  pilot  feedback. 
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Table  2.  Tolerance  Values  for  Derived  Scores 


Base 

Lower 

Upper 

l1  nits 

Final  Approach 

Indicated  Airspeed 

100.0 

-3.0 

5.0 

K  nots 

Centerline 

0.0 

-50.0 

-50.0 

Feet 

Clidepath 

3.863 

-5.5 

+  .55 

Degrees 

Flare 

Airspeed 

BVa 

-3 

+  3 

Knots 

Altitude 

BVb 

-13.0 

13.0 

Feet 

Pitch 

BVC 

-3 

+  3 

Degrees 

Centerline 

0.0 

-15.0 

15.0 

Feet 

Touchdown 

Airspeed 

77.5d 

K  nots 

Vertical  Velocity 

0.0d 

Feet/ 

Sec 

Heading 

30. 2d 

Degrees 

aFlare  Airspeed  was  scored  as  follows:  BV  (Airspeed)  =(-1.96*0  +  98.0 

I  is  lime  in  seconds. 

Flare  Altitude  is  scored  as  follows:  BV  (Altitude  =  (-5.489*0  +  1135.3 
1  Flare  Filch  is  scored  as  follows:  t  from  0  to  8.5  seconds.  BV  (Pitch)  =(.78*t)  -  .59  and  l  after  8.5  seconds.  BV  (Pilch)  = 

r>  5  . 

Touchdown  is  scored  as:  100%  -  (2*ASD)  +  (.04*V\)  -  HD 

ASD  =  airspeed  deviation  from  77.5  knots. 

VV  =  vertical  velocity  at  touchdown. 

HD  =  heading  absolute  deviation  from  302  degrees. 

Procedures 

The  pilots  were  initially  exposed  to  the  simulator  by  giving  them  15  minutes  of  familiarization 
flying  in  a  simulated  Williams  AFB  environment,  during  which  time  they  performed  a  takeoff  and 
closed  pattern  landing,  aerobatic  maneuvers,  and  a  landing  attitude  stall  in  order  to  familarize 
themselves  with  the  handling  characteristics  of  the  simulator.  Each  pilot  was  also  given  an  initial 
training  mission  of  14  landings,  which  required  landing  once  on  each  runway  type  with  and  without 
the  overrun/TD-Zone  lights.  During  the  training  mission,  the  pilots  were  instructed  to  execute 
typical  landings,  based  upon  their  own  judgement  as  instructor  pilots.  They  were  also  encoui  vged  to 
try  to  touchdown  at  about  500  feet  from  the  runway  threshold,  without  distorting  their  normal  flare 
and  landing.  This  touchdown  point  was  in  the  middle  of  the  texture  pattern  area.  During  tra,n'nP- 
the  pilots  were  also  instructed  on  how  to  use  an  in-cockpit  CRT  screen  which  presented  a  simplified 
version  of  the  performance  measures  used  for  scoring  the  landing.  Figure  9  presents  a  typical 
example  of  the  feedback  scoring  for  one  such  landing.  The  basic  purpose  of  this  feedback  scoring  was 
to  maintain  the  motivation  of  the  pilots  during  the  70  landings  required  for  the  experimental 
missions  in  order  to  encourage  a  consistent  level  of  pilot  performance. 

Each  landing  required  approximately  1.5  minutes  from  prior  initialization  to  reinitialization 
after  touchdown.  The  pilots  were  allowed  to  proceed  at  their  own  pare  with  rest  breaks  as  they  were 
required.  Most  of  the  pilots  required  about  two  2 -hour  sessions  on  separate  days  in  order  to  complete 
the  initial  training  and  the  experimental  missions  in  the  AS  FT. 


Analysis 

Table  1  lists  the  performance  measures  which  were  analyzed  for  this  study.  The  basic  design  was 
a  repeated  measures  design  on  all  subjects.  The  data  items  were  first  analyzed  using  multivariate 
analysis  of  variance  techniques  (M  A  NOV  A )  (Rock.  1  ‘>75) .  A  modified  version  of  BM  DX69  (Dixon, 
1970)  was  used  to  perform  the  analyses.  The  multivariate  analyses  were  selected  as  the  appropriate 
omnibus  test  due  to  the  intercorrelation  and  interdependencies  of  the  variables  measured.  Stepdown 
univariate  analysis  of  variance  (ANOVA)  (Myers.  1966)  tests  were  then  computed  in  order  to 
ascertain  the  location  of  the  statistically  significant  differences  within  the  measurement  set.  Tukey 
tests  were  finally  performed  for  investigating  significant  differences  within  specific  variables 
between  the  different  runway  types.  The  criterion  of  a  .05  probability  level  was  used  for  determining 
statistieally  significant  differences  for  the  multivariate,  univariate,  and  Tukey  tests. 


m  RESULTS  AND  DISCUSSION 


The  results  of  the  MANOVAs  and  ANOVAs  are  presented  in  Tables  3,  4,  5,  and  6.  The  four 
different  MANOVA  and  related  ANOVAs  were  performed  in  order  to  illustrate  possible  differences 
in  the  statistical  results  which  might  have  been  due  to  the  particular  subset  of  data  analyzed;  that  is, 
whether  the  night  runways  were  included  with  the  daytime  runways  as  in  Table  3  or  the  two  data  sets 
were  analyzed  separately  as  in  Tables  5  and  6.  Table  4  presents  the  data  separately  analyzed  either 
with  or  without  the  overrun/TD-Zone  lights.  The  MANOVA  for  the  entire  data  set  showed 
statistically  significant  differences  with  probabilities  of  chance  occurrence  less  than  .0001  for  all 
three  independent  variables:  the  runways,  overrun/TD-Zone  lights,  and  subjects.  The  ANOVAs  also 
demonstrated  significant  differences  for  many  of  the  data  items.  The  pilots  were  significantly 
different  on  almost  all  data  items.  Typically,  such  differences  are  found  between  pilots  due  to 
individual  style  differences  (Irish  &  Buckland,  1978).  Some  of  the  univariate  differences  for  runway 
types  are  much  easier  to  interpret  than  others. 


Touchdown  Segment 

The  most  striking  result,  shown  in  Table  7,  concerned  aircraft  vertical  velocity  at  touchdown. 
Aircraft  vertical  velocity  at  touchdown  is  directly  related  to  the  firmness  of  the  landing,  and  pilots 
usually  try  to  minimize  the  firmness  of  the  touchdown  bump.  Although  the  firmness  of  a  touchdown 
is  only  one  aspect  of  a  number  of  parameters  which  are  related  to  the  quality  of  a  particular 
touchdown,  a  major  purpose  of  the  flare  is  to  reduce  aircraft  vertical  velocity  in  a  smooth  fashion  so 
that  aircraft  vertical  velocity  approaches  the  minimum  practical  level  for  a  particular  aircraft  at 
touchdown.  The  optimum  touchdown  vertical  velocity  is  typically  much  less  for  a  small  aircraft  than 
for  a  large  one.  Thus,  studies  of  average  vertical  velocities  at  touchdown  for  commercial  turbojet  air 
transports  (707 -type  aircraft)  found  that  the  average  values  ranged  from  96  to  1 14  ft/min  (Stickle. 
1961,  1962;  Stickle  &  Silsby.  1960).  Vertical  velocity  data  collected  on  T-37  aircraft  in  this  study 
resulted  in  an  average  touchdown  value  of  32  ft/min. 

While  the  simulated  vertical  velocity  was  much  greater  than  the  actual  aircraft  vertical  velocity 
measured  at  Edwards  AFB.  the  texture  patterns  did  produce  an  effect.  The  average  vertical  velocities 
at  touchdown  decreased  systematically  from  195  ft/min  for  the  night  runway  to  147  ftAnin  for  the  4- 
foot  texture  pattern.  These  values  were  also  somewhat  less  than  the  touchdown  vertical  velocities 
reported  for  large  aircraft  simulations.  Various  average  touchdown  vertical  velocities  have  been 
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Table  3.  Results  of  M  ANOVA  and  ANOVA  Analyses 


Data  hem 

Runway 

Probability 

Ovemnt/ 
TD-Zonc  Lights 
Probability 

Subject 

Probability 

Overall  M  ANOVA  (AH  22  variables) 

.000  la 

.0000* 

.0000® 

Final  Approach  Score 

.1046 

.0004® 

.0000® 

Flare  Score 

.2361 

.0460 

.0000® 

Touchdown  Score 

,0000a 

.6155 

.0000* 

Total  Score 

.4163 

.4652 

.0000* 

Final  Approach 

Airspeed  RMS  Deviation 

.1213 

.8282 

.0000® 

Centerline  RMS  Deviation 

,04l7a 

.0001* 

.0000* 

Clidepath  RMS  Deviation 

.0046a 

.0000® 

.0000® 

Flare  Portion 

Airspeed  RMS  Deviation 

.0206a 

.0973 

.0000® 

Altitude  RMS  Deviation 

.0000® 

.4836 

.0000® 

Pitch  RMS  Deviation 

.0000® 

.1591 

.0000® 

Centerline  RMS  Deviation 

.2531 

.0023® 

.0000® 

Start  of  Flare  Altitude 

.0000® 

.0000® 

.0000® 

Touchdown 

Centerline  Position 

.0057® 

.0000® 

.0000® 

Distance  from  Threshold 

.0000® 

.0976 

.0000* 

A  irspeed 

,0000a 

.0658 

.0000* 

Heading 

.0119* 

.8221 

.0000* 

Vertical  Velocity 

.0000® 

.2814 

.0000* 

Smoothness  Profile 

Aileron  Power 

.1216 

.6440 

.0000* 

Roll  RMS  Rate 

.0086® 

.2401 

.0000* 

Elevator  Power 

.0170® 

.1276 

.0000® 

Pitch  RMS  Rate 

.2732 

.0000* 

.0000* 

Rudder  Power 

.2171 

.0481® 

.0000* 

*AI1  variables  wilh  probabilities  less  than  .05  are  considered  to  have  statistically  significant  differences  within  that 
variable  for  that  particular  data  item. 


Table  4.  Results  of  MANOVA  and  ANOVA  Analyses  with  or 
without  Ovenrun/TD-Zone  Lights 


Variable 

Rynway  Probability 
Without  Ove  min/ 
'fD-Zone  Lights 
w 

Runway  Probability 
With  Ove  min 
TD-Zone  Lights 

Overall  MANOVA  (all  22  variables) 

*  .0001 a 

.000  la 

Final  Approach  Score 

.2744 

.0861 

Flare  Score 

.0275a 

.6180 

Touchdown  Score 

.02 1 6a 

.0005® 

Total  Score 

.0857 

.2691 

Final  Approach 

Airspeed  RMS  Deviation 

- 

.7014 

.0232a 

Centerline  RMS  Deviation 

.33% 

.0005® 

Clidepath  RMS  Deviation 

.0262a 

.0454® 

Flare  Portion 

Airspeed  RMS  Deviation 

.0012a 

.7868 

Altitude  RMS  Deviation 

.2631 

.0000® 

Pitch  RMS  Deviation 

.0000a 

.0000® 

Centerline  RMS  Deviation 

.4006 

.4768 

Start  of  Flare  Altitude 

.0714 

.0000® 

Touchdown 

Centerline  Position 

% 

.0936 

.0342® 

Distance  from  Threshold 

.0002 

.0000® 

Airspeed 

,0000a 

.0000® 

Heading 

.0222a 

.1052 

Vertical  Velocity 

,0000a 

.0873 

Smoothness  Profile 

Aileron  Power 

.2538 

.3999 

Roll  RMS  Rate 

.3099 

.0026® 

Elevator  Power 

.1313 

.0770 

Pitch  RMS  Rale 

.3694 

.8790 

Rudder  Power 

T. 

.3787 

.4985 

Note.  —  All  .mibj  rrf  probabilities  were  p  <  .0004  or  less.  M os!  »prr  p  <  .1)000. 

All  variables  with  probabilities  less  than  ,0.»  are  considered  lo  have  slalislieally  significant  differences  within  that 
variable  for  that  particular  data  item. 
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Table  5.  Results  of  MANOVA  and  ANOVA  Analyses  on 
Daytime  Runways  Only 


Runway 

Overrun 

S  ubje  c  t 

Variable 

Probability 

Probability 

Probability 

Overall  MANOVA  (All  22  varia- 


bles) 

,0000a 

,0000a 

.0000® 

Final  Approach  Score 

.3204 

.0035a 

.0000® 

Flare  Score 

.3794 

.0042® 

.0000® 

Touchdown  Score 

.0000“ 

.7464 

.0000® 

Total  Score 

.3035 

.7968 

.0000® 

Final  Approach 

Airspeed  RMS  Deviation 

.2059 

.8550 

.0000® 

Centerline  RMS  Deviation 

.1383 

.0000® 

.0000® 

Glidepath  RMS  Deviation 

.049  la 

.0003® 

.0000® 

Flare  Portion 

Airspeed  RMS  Deviation 

.5823 

.0200® 

.0000® 

Altitude  RMS  Deviation 

.1314 

.1361 

.0000® 

Pitch  RMS  Deviation 

,0000a 

.0219“ 

.0000® 

Centerline  RMS  Deviation 

.2302 

.0163® 

.0000® 

Start  of  Flare  Altitude 

.2062 

.0000® 

.0000® 

Touchdown 

Centerline  Position 

.0310a 

.0000® 

.0000® 

Distance  from  Threshold 

,0000a 

.0167® 

.0000® 

Airspeed 

.0001“ 

.1921 

.0000® 

Heading 

.01 76a 

.4611 

.0000® 

Vertical  Velocity 

,0000a 

.1170 

.0000® 

Smoothness  Profile 

Aileron  Power 

.2891 

.7434 

.0000® 

Roll  RMS  Rate 

.2620 

.0993 

.0000® 

Elevator  Power 

.026  la 

.0654 

.0000® 

Pitch  RMS  Rate 

.4083 

.0000® 

.0000® 

R  udder  Power 

.2349 

.0310® 

.0000® 

8 A II  variables  with  probabilities  less 

than  .OS  are  considered  to  have 

statistically  significant 

differences  within  that 

variahle  (or  that  particular  data  ilrm. 
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Table  6.  Results  of  MANOV  A  and  ANOVA  Analyses  for 
Nighttime  Runway  Only 


Variable 

TD-Zone  Lights 
Probability 

Subjects 

Overall  MANOV  A  (ail  22  variables) 

.0001a 

.0000“ 

Final  Approach  Score 

.0284a 

.0000“ 

Flare  Score 

.0395* 

.0000* 

Touchdown  Score 

.S208 

.0398“ 

Total  Score 

.01 13a 

.0000* 

Final  Approach 

Airspeed  RMS  Deviation 

.2997 

.0000“ 

Centerline  RMS  Deviation 

.1468 

.0000“ 

Glidepath  RMS  Deviation 

.0219* 

.0000“ 

Flare  Portion 

Airspeed  RMS  Deviation 

.1382 

.0061“ 

Altitude  RMS  Deviation 

.2934 

.0007“ 

Pitch  RMS  Deviation 

,0154a 

.0000“ 

Centerline  RMS  Deviation 

,0266a 

.0011“ 

Start  of  Flare  Altitude 

.7715 

.0004“ 

Touchdown 

(.enterline  Position 

.0024® 

.0002“ 

Distance  from  Threshold 

.0643 

.0001“ 

Airspeed 

.0924 

.0000“ 

Heading 

.0086“ 

.0218“ 

Vertical  Velocity 

.1685 

.5052 

Smoothness  Profile 

Aileron  Power 

.6999 

.0002“ 

Roll  RMS  Rate 

.3193 

.0001* 

Elevator  Power 

.7835 

.0309* 

Pitch  RMS  Rate 

.1056 

.0012* 

Rudder  Power 

.5763 

.1206 

aAII  variable  with  probabilities  less  than  .05  are  considered  to  have  statistteaUy  significant  differences  within  that  variable  for 
that  particular  data  item. 
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Table  7.  Vertical  Velocity  (Ft/Min,  a  Negative  Value)  at  Touchdown 


ASPT  Simulated 

Visual  Scene 

Without  Overrun/ 
TD-Zone  Lights 

With  Overrun/ 
TD-Zone  Lights 

Average® 

Night 

201 

190 

195 

Bare  Bones 

195 

171 

183 

Willie 

172 

179 

175 

25  Ft  Texture 

168 

168 

168 

16  Ft  Texture 

158 

173 

165 

8  Ft  Texture 

151 

170 

161 

4  Ft  Texture 

136 

158 

147 

aANOVA  p  <  .0000. 

Results  of  Tukey  Tests 

Bare 

25  Ft 

16  Ft 

8  Ft 

4  Ft 

Bones 

Willie  Texture 

Texture  Texture 

Texture 

Night 

NS 

NS  * 

* 

* 

* 

Bare  Bones 

NS  NS 

NS 

* 

* 

Willie 

NS 

NS 

NS 

* 

25  Ft  Texture 

NS 

NS 

NS 

16  Ft  Texture 

NS 

NS 

8  Ft  Texture  NS 

Note.  —  Averaged  Vertical  Velocity  of  actual  aircraft  at  touchdown  was  32  ft/min.  based  on  Edwards  Flight  Test  Center  Cine- 
Theodolite  Tracking  of  T-37  (74  landings) 

•  —  Statistically  significantly  different  at  .05  level. 

NS  —  Not  significantly  different. 

reported  for  transport  visual  flight  simulators,  ranging  from  216  to  258  ft/min  (Chase,  1971)  to  510 
to  960  ft/min  (Crane,  1962). 

The  second  variable  of  the  overrun/TD-Zone  lights  appeared  to  disrupt  the  linear  decrease  in 
touchdown  vertical  velocity.  Without  the  overrun,  the  linear  trend  in  decreasing  vertical  velocity  at 
touchdown  was  apparent.  With  the  overrun  present,  the  range  of  touchdown  vertical  velocities  was 
reduced,  and  the  linear  decrease  in  vertical  velocity  was  disrupted.  There  was  no  readily  apparent 
statistical  interaction  between  the  runway  overrun  and  texture  patterns.  The  presence  of  the  overrun 
appeared  to  make  performance  on  all  the  runways  more  similar,  apparently  somewhat  offsetting  the 
influence  of  the  texture  patterns  on  the  runway. 

However,  the  separate  analyses  performed  on  the  data  with  or  without  the  overrun/TD-Zone 
light  variable  still  showed  several  significant  differences  under  both  conditions.  Table  4  shows  the 
results  of  these  analyses.  Although  there  are  some  differences  in  the  pattern  of  significant  differences 
between  the  two  conditions,  all  but  two  differences  fall  within  the  overall  pattern  of  statistically 
significant  differences  shown  in  Table  3  for  the  combined  data.  The  two  additional  significant 
differences  are  the  flare  score  without  the  overrun/TD-Zone  lights  (p  <0275)  and  the  final  approach 
airspeed  RMS  deviation  with  the  overrun/TD-Zone  lights. 


Tukey  tests  were  performed  at  the  .05  level  to  compare  statistical  differences  between  individual 
runways  for  the  touchdown  vertical  velocity.  Table  7  also  displays  the  pattern  of  Tukey  lest 
differences  which  was  found  for  vertical  velocity  at  touchdown.  In  general,  if  a  runway  is  compared 
to  the  next  most  similar  runway,  as  they  are  arranged  in  Table  7,  it  was  not  statistically  significantly 
different.  However,  runway  types  which  were  two  or  three  types  removed  from  each  other  were 
significantly  different.  Thus,  the  average  performance  differences  from  one  level  of  texture  to 
another  were  not  sufficient  to  be  significantly  different,  due  to  the  variability  of  the  data.  However, 
there  was  a  readily  apparent  trend  in  the  decreasing  vertical  velocity  at  touchdown  from  the  night 
runway  to  the  most  highly  textured  (4-foot)  daytime  runway. 

While  there  were  significant  differences  for  several  of  the  other  data  parameters,  the 
interpretation  of  the  differences  is  not  as  dear  as  it  is  for  the  aircraft  vertical  velocity  at  touchdown. 
Consistent  trends  which  varied  along  an  apparent  dimension  of  runway  texturing  did  not  appear.  For 
example,  the  touchdown  score,  which  was  a  composite  or  derived  score  based  upon  touchdown 
vertical  velocity,  airspeed,  and  heading,  showed  statistically  significant  differences  across  runway 
types  (p  <.0000).  However,  the  Tukey  tests  of  touchdown  vertical  velocity  indicated  that  only  the 
score  from  the  Bare  Bones  runway  was  significantly  different  from  those  of  other  runway  types. 
Table  8  shows  the  touchdown  scores  for  the  different  runway  conditions.  The  score  on  the  Bare 
Bones  runway  was  lower  than  the  scores  for  the  other  runways.  Apparently  the  combination  of  high 
airspeed  and  vertical  velocity  at  touchdown  significantly  lowered  the  touchdown  scores  for  the  Bare 
Bones  runway  versus  the  other  runways.  Table  9  shows  the  airspeed  values  at  touchdown  for  the 
various  experimental  conditions.  While  the  night  runway  produced  the  highest  vertical  velocities  at 
touchdown,  the  airspeeds  were  significantly  lower  at  touchdown.  Table  10  presents  the  data  on 
heading  at  touchdown.  Both  the  night  and  the  Bare  Bones  runways  produced  touchdown  headings 
closer  to  the  ideal  heading  of  302  degrees:  however,  this  effect  was  apparently  not  large  enough  to 
greatly  influence  the  touchdown  scores  for  the  two  runway  types. 

The  position  of  the  aircraft  relative  to  the  centerline  at  touchdown  also  showed  significant 
differences  across  runway  types.  This  effect  is  shown  in  Table  1 1.  In  general,  the  pilots  were  more 
accurate  in  positioning  the  aircraft  on  the  runway  centerline  of  the  more  textured  runways.  The 
distance  from  the  runway  threshold  to  aircraft  touchdown  also  showed  significant  differences  across 
runway  types.  Table  12  shows  the  average  values  for  the  runway  types  and  illustrates  the  pattern  of 
significant  differences  from  the  Tukey  tests.  There  was  a  tendency  to  touch  down  sooner,  i.e..  closer 
to  the  runway  threshold,  for  the  Bare  Bones  runway,  and  farther  from  the  threshold  for  the  night 
runway. 


Flare  Segment 

Although  the  night  runway  landings  were  slightly  farther  down  the  runway  than  the  day 
landings,  the  start  of  flare  altitude  (1000  feet  from  runway  threshold)  was  slightly  higher.  Table  13 
shows  the  average  start  of  flare  altitudes.  The  primary  difference  found  by  the  Tukey  tests  was 
between  the  night  and  the  day  runways. 

Three  of  the  four  measures  on  the  flare  segment  were  statistically  significantly  different  for  the 
average  scores:  the  R  M  S  deviation  scores  for  airspeed  (p  <0206) . altitude  (p  <0000). and  pitch  (p  < 
.0000).  Tables  14.  15.  and  16  show  the  average  values  and  T  ukey  test  results  for  these  RMS  measures. 
The  flare  section  centerline  RMS  deviation  was  not  statistically  significantly  different  across  runway 
types. 


Table  8.  Touchdown  Score  (%) 


Table  10.  Heading  at  Touchdown 


Table  11.  Centerline  Position  (Feet)  at  To 


S  Not  significantly  different. 


S  Not  significantly  different. 


Table  15.  Flare  Altitude  RMS  Deviation  (Feet) 


The  main  differences  for  the  flare  airspeed  RMS  deviation  seem  to  lie  between  performance  on 
the  night  runway  versus  the  other  runways,  with  the  RMS  deviation  being  lower  for  the  night 
runway.  Apparently  the  pilots  maintained  airspeeds  for  the  night  runways  which  were  closer  to  the 
“ideal  flare  airspeed  profile.  This  findings  is  in  agreement  with  the  lower  airspeeds  for  aircraft 
touchdown  at  night  shown  in  Table  9.  The  night  runway  touchdown  airspeeds  were  significantly 
lower  than  those  for  the  other  runways,  while  the  Bare  Bones  touchdown  airspeeds  were  statistically 
significantly  higher  at  touchdown. 

Table  15  shows  that  the  RMS  deviation  from  the  “ideal’'  flare  altitude  profile  was  greater  for  the 
night  runway.  The  T ukey  tests  results  indicate  that  this  was  the  primary  significant  difference  for  this 
variable.  This  result  is  in  agreement  with  the  data  for  the  start  of  flare  altitude  (see  Table  1  1)  which 
indicated  that  the  pilots  were  higher  at  the  start  of  flare  for  the  night  runway  scene. 

The  results  for  flare  pitch  RMS  deviation  are  presented  in  Table  16.  Apparently  the  flare  pitch 
R  MS  deviation  from  the  “ideal"  flare  pitch  profile  was  larger  for  the  night  runway  and  smaller  for 
the  Bare  Bones  runway  versus  the  other  daytime  runways.  The  results  of  the  Tukey  tests  indicate 
that  both  the  night  and  the  Bare  Bones  runways  were  statistically  significantly  different  from  each 
other  and  from  the  other  runways  in  this  regard. 

The  derived  flare  score  was  not  significantly  different  across  runway  tvpes  on  the  average: 
however,  it  was  significantly  different  when  the  scores  were  analyzed  for  onlv  the  data  without  the 
overrun/Tf)-Zone  lights.  Table  17  shows  the  average  values  for  these  data.  Apparently  the  flare 
scoring  was  influenced  by  the  runway  types  without  the  presence  of  the  overrun,  but  the  presence  of 
the  overrun/1  D-Zone  lights  produced  more  homogeneous  scores,  in  a  fashion  which  was  similar  to 
the  influence  of  the  overt  u.i  on  vertical  velocity  at  touchdown.  The  only  significant  difference  for 
the  landings  without  the  .<v  mm/TD-Zonc  lights,  which  was  demonstrated  by  the  Tukey  tests,  was 
between  the  night  and  the  1-foot  texture  patterns.  In  this  no-lights  condition,  the  texture  patterns 
apparently  produced  poorer  flare  scores.  Most  of  the  pilots  indicated  dissatisfaction  with  the  flare 
scoring,  (.onsequently.  they  were  instructed  to  execute  their  flares  and  landings  according  to  their 
own  best  judgement  and  ignore  the  flare  scoring,  because  it  was  onlv  an  initial  attempt  at  scoring 
flare  performance. 

Final  Approach  Segment 

Performance  differences  were  also  found  in  the  final  approach  segment  for  the  centerline  (p  < 
.0474)  and  glidepath  (p  <0040)  RMS  deviat  ions.  The  average  scores  and  Tukey  test  results  for  the 
centerline  RMS  deviations  are  shown  in  Table  18.  There  appears  to  be  slightly  greater  centerline 
deviation  for  the  night  runway,  especially  when  the  night  runway  was  compared  to  the  Bare  Bones. 
Millie,  and  8-foot  textured  runways.  None  of  the  Tukey  test  results  were  significantly  different, 
apparently  because  the  Tukey  test  is  more  statistically  conservative  than  the  overall  ANOVA.  The 
average  values  and  Tukey  test  results  for  the  final  approach  glidepath  RMS  deviation  are  shown  in 
Table  19.  Again,  the  main  differences  appeared  to  be  between  the  night  runway  and  the  davtime 
runways.  Average  values  for  the  final  approach  airspeed  RMS  deviation  were  only  statistically 
significantly  different  (p  <.0232)  for  the  runways  with  the  overrun/TD-Zone  lights.  The  average 
values  and  Tukey  test  results  are  shown  in  Table  20. 

Only  two  of  five  smoothness  profile  measures,  roll  RMS  rate  and  elevator  power,  differed 
significantly  across  runway  types.  The  roll  RMS  rate  data  are  shown  in  Table  21.  The  primary 
difference  was  hetween  the  night  runway  and  the  daytime  visual  scenes.  The  elevator  power  data, 
shown  in  Table  22.  also  indicate  a  slight  increase  in  control  activity  for  the  night  runway.  Like  the 


Not  significantly  different. 


Table  19.  Final  Approach  Glide  Path  RMS  Deviation  (Degrees) 


Table  20.  Final  Approach  Airspeed  (Knots)  RMS  Deviation 


Table  21.  Roll  RMS  Rate  (Degrees/Second) 


Table  22.  Elevator  Power  (Pounds-Degrees/Second) 


roll  RMS  rate  data,  the  primary  differences  in  the  elevator  power  data  are  between  the  night  and  the 
daytime  runways.  These  night  versus  daytime  differences  are  in  agreement  with  the  similar 
differences  found  for  the  final  approach  centerline  and  glidepath  deviations.  The  night  visual  scene 
was  more  panoramic  in  terms  of  the  point  lights  on  the  ground  and  most  of  the  pilots  expressed  a 
preference  for  the  night  scene  because  of  the  more  distributed  visual  scene.  Possibly  the  night  scene 
provided  better  cues  for  initial  manual  course  corrections  than  the  day  scenes,  resulting  in  more 
control  activity  and  greater  centerline  and  glidepath  RMS  deviations. 

The  presence  or  absence  of  the  TD-Zone  lights  produced  some  effects  on  the  night  runway 
performance;  however,  these  effects  were  not  in  the  expected  direction.  The  data  items  which  were 
statistically  significantly  different  are  shown  in  Table  6.  The  average  for  these  data  items  are  shown 
in  Table  23.  Tukey  tests  were  not  employed  because  there  were  only  two  levels  for  this  variable.  In 
comparing  the  average  differences  for  the  eight  significantly  different  variables,  the  performance  was 
consistently  better  for  the  night  runway  without  the  TD-Zone  lights.  The  final  approach,  flare,  and 
total  scores  were  higher  without  the  TD-Zone  lights.  The  RMS  deviations  for  the  final  approach 
glidepath,  flare  pitch,  and  flare  centerline  were  smaller,  and  thus  better,  for  the  night  scene  without 
the  TD-Zone  lights.  The  average  distance  from  the  centerline  at  touchdown  was  smaller  or  better  for 
the  scene  without  the  TD-Zone  lights.  The  aircraft  heading  at  touchdown  was  also  closer  to  the  ideal 
of  302  degrees  for  the  scene  without  TD-Zone  lights.  It  was  anticipated  that  the  TD-Zone  lights 
would  provide  additional  runway  texturing  and  thereby  produce  effects  similar  to  the  texturing  of 
the  day  runways.  However,  although  this  variable  did  produce  lower  average  vertical  velocity  at 
touchdown,  the  effect  was  not  statistically  significantly  different.  The  only  statistically  significant 
effects  indicated  poorer  performance  during  the  final  approach,  flare,  and  landing  when  the  TD- 
Zone  lights  were  present. 


Table  23.  Significantly  Different  Variable  Averages 
for  Night  Runway  with  and  without  TD-Zone  Lights 


Variable 

Without 

TD-Zone  Lights 

With _ 

TD-Zone  Lights 

Final  Approach  Score  (%) 

80.1 

69.2 

Flare  Score  (%) 

52.4 

42.1 

Total  Score  (%) 

73.0 

66.2 

Final  Approach 

Glidepath  RMS  Deviation  (Degrees) 

.354 

.477 

Flare  Portion 

Pitch  RMS  Deviation  (Degrees) 

4.28 

4.84 

Centerline  RMS  Deviation  (Feet) 

6.39 

8.26 

Touchdown 

Centerline  Position  (Feet) 

-3.63 

-5.27 

Heading  (Degrees) 

302.0 

301.7 
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The  presence  or  absence  of  the  runway  overrun  produced  many  statistically  significant  effects 
on  the  daytime  runways,  as  shown  in  Table  5.  Table  24  contains  the  average  values  for  the 
statistically  significant  data  items.  There  is  no  clear  pattern  in  the  significant  differences  which  were 
found.  For  example,  the  final  approach  score  was  better  without  the  overrun,  but  the  flare  score  was 
better  with  the  overrun.  The  final  approach  centerline  RMS  deviation  was  better  with  the  overrun, 
but  the  glidepath  RMS  deviation  was  better  without  the  overrun.  The  flare  RMS  deviations  for 
airspeed  and  pitch  were  better  with  the  overrun,  but  the  centerline  RMS  deviation  was  better 
without  the  overrun.  The  touchdown  centerline  position  was  better  without  the  overrun,  but  the 
distance  from  the  threshold  was  closer  to  the  requested  touchdown  point  with  the  overrun.  The  pitch 
RMS  rate  was  slightly  better  with  the  overrun,  but  the  rudder  power  was  slightly  better  without  the 
overrun.  Thus,  the  individual  effects  produced  by  the  overrun  variable  were  equivocal  at  best. 
However,  while  the  overrun  did  not  produce  a  statistically  significant  effect  on  the  vertical  velocity 
at  touchdown,  it  did  influence  the  range  of  the  vertical  velocities  that  were  produced  by  the  different 
runway  textures.  Table  7  shows  this  effect.  Apparently  the  presence  of  the  overrun  restricted  the 
range  of  vertical  velocities  produced  by  the  texture  patterns.  Possibly  the  pilots  did  not  use  the  visual 
texture  pattern  information  as  much  when  an  overrun  was  present. 


Variable 


Table  24.  Significantly  Different  Variable  Averages 
for  Daytime  Runways  with  and  without  Runway  Overrun 


Without 

Overrun 


With 

Overrun 


Final  Approach  Score  (%) 

Flare  Score  (%) 

Final  Approach 

Centerline  RMS  Deviation  (Feet) 
Glidepath  RMS  Deviation  (Degrees) 

Flare  Portion 

Airspeed  RMS  Deviation  (Knots) 
Pitch  RMS  Deviation  (Degrees) 
Centerline  RMS  Deviation  (Feet) 

Start  of  Flare  Altitude  (Feet) 

Touchdown 

Centerline  Position  (Feet) 

Distance  from  Threshold  (Feet) 

Smoothness  Profile 

Pitch  RMS  Rate  (Degrees/Second) 

Rudder  Power  (Lb- Degrees/Second) 


83.3 

39.2 


19.0 

.32 


3.10 

3.64 

7.33 

1,446.9 


-2.83 

539.8 


.74 

.006 


77.4 

45.8 


15.7 

.38 


2.79 

3.38 

8.17 

1,440.2 


-4.27 

511.1 


.70 

.010 
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IV.  CONCLUSIONS 


The  primary  effect  of  the  runway  texture  patterns  was  observed  on  the  aircraft  vertical  velocity 
at  touchdown.  This  parameter  is  the  primary  indicator  of  touchdown  firmness,  and  pilots  typically 
try  to  minimise  the  firmness  of  the  touchdown  bump.  Thus,  vertical  velocity  is  a  good  indicator  of 
the  pilots’  ability  to  detect  and  thus  control  their  rale  of  closure  with  the  runway.  Although  the 
runway  texture  patterns  did  not  produce  vertical  velocities  as  low  as  those  in  the  actual  aircraft,  they 
did  influence  the  touchdown  vertical  velocities  in  a  consistent  manner.  The  addition  of  the  texture 
patterns  did  systematically  reduce  the  simulated  aircraft  vertical  velocity  at  touchdown  from  201  ft/ 
min  for  the  night  runway  to  136  ft/min  for  the  4-foot  texture  pattern.  The  presence  of  the  TD-Zone 
lights  in  the  night  runway  reduced  the  vertical  velocity  at  touchdown,  in  a  fashion  similar  to  the 
texture  patterns  on  the  daytime  runways.  However,  this  effect  was  not  statistically  significant. 

The  overrun  with  its  chevron  markings  also  provided  a  visual  textural  cue,  which  appeared  to 
provide  an  alternate  textural  cue  to  the  runway  patterns.  The  presence  of  the  overrun  compressed 
the  range  of  the  touchdown  vertical  velocities  across  runway  types,  reducing  the  size  of  the 
differences  between  runways.  Apparently  there  was  a  tendency  for  the  pilots  to  use  primarily  the 
overrun  visual  cues  when  they  were  present  and  not  to  use  the  runway  texture  patterns.  Thus, 
average  touchdown  vertical  velocities  for  the  Bare  Bones  runway  were  lower  when  the  overrun  was 
used;  whereas,  the  average  touchdown  vertical  velocities  for  the  4-foot  textured  runway  were  higher 
when  the  overrun  was  used. 

All  of  the  simulated  landings  produced  vertical  velocities  which  were  much  higher  than  those 
measured  for  the  actual  T-37  aircraft,  using  the  same  pilots,  at  the  Air  Force  Flight  Test  Center. 
Thus,  the  visual  texture  cues,  the  texture  patterns  and  the  runway  overrun,  which  were  used  for  this 
study  did  not  eliminate  the  problem  of  excessive  vertical  velocity  at  touchdown  in  flight  simulators. 
While  these  visual  texture  cues  did  influence  and  systematically  reduce  simulated  aircraft  vertical 
velocity  at  touchdown,  it  appears  that  other  visual  cues  were  still  missing,  possibly  including  other 
textural  cues.  One  cue  which  was  eliminated  on  purpose  was  vertical  object  cues  near  the  runway. 
This  was  done  in  order  to  control  for  this  possible  conflicting  variable.  That  is,  the  pilots  might  have 
used  such  vertical  object  cues  for  some  runway  types,  but  not  for  other  runways.  Pilots  typically  use 
the  best  cues  available  for  any  particular  flying  task,  with  the  use  of  particular  environmental 
information  cues  varying  according  to  which  cues  are  available.  Further  studies  of  visual  cue 
requirements  for  landing  simulators  should  definitely  test  the  efficacy  of  vertical  object  cues. 

An  additional  area  for  visual  textural  cues  might  be  along  the  sides  of  the  runway.  Pilots 
typically  report  that  they  do  not  look  at  the  runway  immediately  in  front  of  them  during  the 
landing;  rather,  they  look  at  the  end  of  the  runway  and  the  horizon.  Harris,  Waller,  and  Salmirs 
(1978)  reported  in  a  study  using  the  Langley  Research  Center  occulometer  that  when  pilots  were 
using  a  heads-down  cathode  ray  tube  approach  and  landing  display,  they  generally  did  look  close  to 
the  horizon  during  the  flare.  Thus,  it  is  possible  that  pilots  are  using  peripheral  vision  cues  to 
monitor  forward  motion  and  vertical  motion  (vertical  velocity)  during  the  aircraft  flare.  Harris  et  aL 
(1978)  also  used  checkerboard  patterns  either  on  the  runway  or  external  to  the  runway.  These 
patterns  were  either  38  (124.7  ft),  76  (249.3  ft),  or  152  (498.7  ft)  meters  on  a  side  on  the  runway  or 
228  (748  ft)  meters  on  a  side  external  to  the  runway.  These  checkerboard  patterns  were  thus  much 
larger  than  the  textural  patterns  used  in  the  current  study.  Harris  et  at.  (1978)  reported  that  the 
checkerboard  patterns  on  the  runway  produced  a  slight  improvement  in  touchdowB  performance 
and  an  increase  in  the  number  of  column  control  inputs.  The  checkerboard  patterns  external  to  the 
runway  did  not  improve  performance.  In  fact,  the  “touchdown  performance  vertical  speed  and 
airspeed  without  the  fields  was  somewhat  more  indicative  of  a  flare,”  and  Harris  et  al.  (1978) 
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concluded  that  “the  addition  of  external  fields  are  not  conducive  to  good  flare  and  landing 
performance  with  this  display.”  Thus,  based  upon  Harris’  use  of  relatively  large  external 
checkerboard  patterns,  questions  regarding  the  utility  of  visual  textural  cues  external  to  the  runway 
itself  are  still  open  to  further  research.  Perhaps  smaller  checkerboard  patterns  would  be  more  useful 
than  the  large  patterns  used  by  Harris. 

The  influence  of  the  different  runway  types  on  the  other  data  parameters  which  were  analyzed 
was  not  as  obvious.  The  most  wide-spread  differences  occurred  between  the  night  runway  scene  and 
the  daytime  runways.  The  widespread  and  somewhat  pervasive  differences  found  between  the  night 
and  the  daytime  runways  provide  some  factual  data  to  support  the  concept  that  pilots  do  employ 
somewhat  different  techniques  for  landing  at  night.  This  possible  difference  between  pilot  landing 
techniques  for  night  versus  day  scenes  should  be  investigated  further  since  many  flight  simulators  are 
now  starting  to  use  night-only  visual  simulation  systems  for  pilot  training.  This  day/night  difference 
may  be  difficult  to  find  in  pilot  training  now,  due  to  the  general  inadequacy  of  visual  flight  simulators 
for  training  flare  and  touchdown.  However,  it  may  be  more  important  for  future  flying  training  as 
the  visual  scenes  are  improved,  and  they  are  actively  used  for  training  the  flare  and  landing. 

The  use  of  simulated  texture  patterns  on  the  runway  touchdown  zone  area  did  improve  pilot 
performance  in  terms  of  vertical  velocity  (sink  rate)  at  touchdown.  This  variable  alone  was  not 
sufficient  to  produce  touchdown  vertical  velocities  in  the  range  of  actual  aircraft  performance,  and 
the  edge  capacity  of  our  current  system  did  not  permit  us  to  study  other  visual  cues  at  the  same  time. 
It  will  probably  be  necessary  to  improve  the  visual  texture  cues  as  well  as  the  other  typical  visual 
cues,  which  are  used  in  flight  simulators,  in  order  to  solve  the  general  problem  of  excessive  vertical 
velocity  at  touchdown.  Such  other  visual  cues,  which  will  be  investigated  in  later  studies,  concern 
vertical  object  cues  near  the  runway,  textural  cues  adjacent  to  the  runway  and  depth  cues  related  to 
collimated  image  techniques. 


V.  SUMMARY 

The  effects  of  seven  different  runway  types  were  investigated  on  pilot  performance  during 
landings  in  a  T-37  flight  simulator.  Data  were  also  gathered  on  six  of  the  same  12  pilots  during  actual 
T-37  aircraft  landings  at  the  Air  Force  Flight  Test  Center.  The  seven  simulated  runways  consisted  of 
one  night  runway  and  six  day  runways  with  varying  amounts  of  textural  information  cues  on  the 
runway  touchdown  zone  area.  The  night  runway  was  also  tested  with  and  without  TD-Zone  landing 
lights  and  the  day  runways  were  tested  with  and  without  the  runway  overrun.  The  simulated  aircraft 
average  vertical  velocity  at  touchdown  decreased  systematically  from  201  ft/min  for  the  night 
runway  without  the  TD-Zone  lights  to  136  ft/min  for  the  day  runway  with  4-foot  texture  patterns. 
The  day  runways  alone  without  the  overrun,  varied  from  195  ft/min  for  the  Bare  Bones  runway  to 
136  ftAnin  for  the  4-foot  texture  pattern.  Although  these  average  vertical  velocities  were  still  much 
higher  than  those  recorded  in  the  actual  aircraft  (32  ft/min),  the  texture  patterns  did  influence  the 
pilot  flare  and  touchdown  in  a  systematic  manner.  Additional  visual  cues  probably  would  have 
reduced  the  vertical  velocities  even  more,  but  the  limited  edge  capacity  of  the  C1G  scene  used  here 
did  not  permit  a  study  of  other  visual  cues  while  investigating  texture  patterns.  The  presence  of  the 
TD-Zone  lights  in  the  night  scene  also  reduced  the  average  vertical  velocity  at  touchdown  (190  ft/ 
min),  but  this  difference  was  not  statistically  significant.  The  presence  of  the  runway  overruns  on  the 
daytime  runways  limited  the  overall  range  of  touchdown  vertical  velocities  to  a  smaller  range, 
spanning  from  176  ftAnin  for  the  Willie  runway  to  158  ft/min  for  the  4-foot  textured  runway.  When 
the  overrun  was  present,  apparently,  the  pilots  used  the  overrun  visual  cues,  the  chevron  texture 
patterns  and  other  related  cues,  in  addition  to  the  runway  texture  patterns  in  order  to  perform  the 
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flare  and  touchdown.  This  resulted  in  reduced  overall  touchdown  vertical  velocities,  but  apparently 
the  more  uniform  pilot  performances  (restricted  range)  did  not  involve  an  optimum  use  of  the  4-foot 
texture  patterns.  Several  other  data  parameters  also  varied  across  runway  types;  however  there  were 
no  consistent  differences  related  to  runway  texture  patterns.  The  significant  effects  with  the  other 
data  parameters  were  most  often  related  to  differences  between  the  night  and  the  day  runway  scenes. 


REFERENCES 


Armstrong,  B.D.  Difficulties  with  the  simulation  of  aircraft  landings.  Technical  Report  68 
116,  AD-840  624.  Royal  Aircraft  Establishment,  Farnborough,  England,  May  1968. 

Armstrong,  B.D.  Flight  trials  to  discover  whether  peripheral  vision  is  needed  for  landing. 
Technical  Report  70  205,  AD-724  009.  Royal  Aircraft  Establishment,  Farnborough, 
England,  November  1970 

BeD,  F.E.  Advanced  simulation  in  undergraduate  pilot  training  ( ASUPT ),  Technical  Fact 
Sheet.  AFHRL-TN -73-01.  Williams  AFB,  AZ :  Flying  Training  Division,  Air  Force  Human 
Resources  Laboratory,  May  1974. 

Bock,R.D.  Multivariate  statistical  methods  in  behavioral  research.  New  York:  McGraw-Hill, 
1975. 

Bray,  RjS.  Vertical  motion  requirements  for  landing  simulation.  NASA  TMX-62236,  N73- 
17020.  Washington,  D.C.:  National  Aeronautics  and  Space  Administration,  February  1973. 

Chase,  W.D.  Evaluation  of  several  TV  display  systems  for  visual  simulation  of  the  landing 
approach.  NASA-TN-D-6274,  N71-22270.  Washington,  D.C.:  National  Aeronautics  and 
Space  Administration,  March  1971. 

Crane,  H.L.  Preliminary  evaluation  of  two  jet-transport  simulators  for  the  investigation  of 
landing-contact  conditions.  N ASA-TN-D-1495.  Washington,  D.C.:  National 
Aeronautics  and  Space  Administration,  November  1962. 

Dixon,  WJ.  (Ed.).  BMD,  Biomedical  Computer  Programs,  X-Series  Supplement.  Los 
Angeles:  University  of  California  Press,  1970. 

Dyda,  ILL,  A  Lew,  D.W.  An  investigation  to  determine  validity  of  using  a  simulator  to 
predict  landing  impact  characteristics,  program  summary.  ASD-TDR-63-71 1, 
Volume  I.  Wright-Patterson  AFB,  OH:  Aeronautical  Systems  Division,  August  1963. 

Hanis,  RX»,  Waller,  M.C.,  A  Sabnira,  S.  Runway  texturing  requirements  for  a  head-down 
cathode  ray  tube  approach  and  landing  display.  Paper  presented  at  AIAA  Flight 
Simulation  Technologies  Conference,  Arlington,  Texas,  September  18-20,  1978. 

Irish,  P. A.,  A  Buckland,  G.H.  Effects  of  platform  motion,  visual  and  G-seat  factors  upon 
experienced  pilot  performance  in  the  flight  simulator.  AFHRL-TR-78-9,  AD-A055 
691.  Williams  AFB,  AZ:  Flying  Training  Division,  Air  Force  Human  Resources 
Laboratory,  June  1978. 


'  .-tVA,  n L  vbrafdftt- 


Sorum,  D.R.,  A  Fisler,  B.L.  MAC  operational  testand  evaluation  of  C-5/C1 4 1  limited  visual 
system.  Final  Report,  AD  B000412L.  Hq  MAC,  Scott  AFB,  IL:  Headquarters  Military 
Airlift  Command,  November  1974. 

Stickle,  J.W.  An  investigation  of  landing-contact  conditions  for  two  large  trubojet 
transports  and  a  turboprop  transport  during  routine  daylight  operations.  NASA* 
TN-D-899,  Washington,  D.C.:  National  Aeronautics  and  Space  Administration,  May  1961. 

Stickle,  J.W.  An  investigation  of  landing-contact  conditions  for  several  trubojet  transports 
during  routine  daylight  operations  at  New  York  International  Airport.  NASA-TN- 
D-1483.  Washington,  D.C.:  National  Aeronautics  and  Space  Administration,  October  1962. 

Stickle,  J.W.,  A  Sibby,  N.S.  An  investigation  of  landing-contact  conditions  for  a  large 
turbojet  transport  during  routine  daylight  operations.  NASA-TN-D-527. 
Washington,  D.C.:  National  Aeronautics  and  Space  Administration,  October  1960. 

Valverde,  H.H.  Flight  simulators— A  review  of  the  research  and  development.  AMRL-TR- 
68-97,  AD-855  582.  Wright-Patterson  AFB,  OH:  Aeromedical  Research  Laboratory,  July 
1968 

Wempe,  T.,  A  Palmer,  E.  Pilot  performance  with  a  simulator  pictorial  landing  display  including 
different  conditions  of  resolution  and  update  rale.  Proceedings  of  the  Sixth  Annual 
Conference  on  Manual  Control,  W  right -Patterson  AFB,  OH-.  Air  Force  Institute  of 
Technology,  April  7-9,  1970,  47-81. 


SUPPLEMENTARY 


INFORMATION 


pB)-fl-o?c?y-3¥- \ 

o 


WLI  TO 
ATTN  OF: 

SUBJECT: 


TSR 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  FORCE  HUMAN  RESOURCES  LABORATORY  (AFSCl 
BROOKS  AIR  FORCE  BASE.  TEXAS  78235 


lI 


1  6  JAN  1981 


Removal  of  Export  Control  Statement 


Defense  Technical  Information  Center 
Attn:  DTIC/DDA  (Mrs  Crumbacker) 

Cameron  Station 
Alexandria  V A  22314 

1.  Please  remove  the  Export  Control  Statement  which  erroneously  appears  on 
the  Notice  Page  of  the  reports  listed  8<aMlai6S&fiESMIBPfc .  This  statement  is 
intended  for  application  to  Statement  B  reports  only. 

2.  Please  direct  any  auestions  to  AFHRL/TSR,  AUTOVON  240-3877. 

FOR  THE  COMMANDER 

WENDELL  L.  ANDERSON,  Lt  Col,  USAF  1  Atch 

Chief,  Technical  Services  Division  List  of  Reports 


Cy  to:  AFHRL/TSE 


